A comprehensive summary informativeness evaluation for RST-based summarization methods

Motivated by governmental, commercial and academic interests, automatic text summarization area has experienced an increasing number of researches and products, which led to a countless number of summarization methods. In this paper, we present a comprehensive comparative evaluation of the main automatic text summarization methods based on Rhetorical Structure Theory (RST), claimed to be among the best ones. Additionally, we test machine learning techniques trained on RST features. We also compare our results to superficial summarizers, which belong to a paradigm with severe limitations, and to hybrid methods, combining RST and superficial methods. Our results show that all RST methods have similar overall performance and that they outperform the superficial methods. In terms of precision, the method we propose is the best one, while it competes with other ones for coverage. Machine learning techniques achieved high accuracy in the classification of text segments worth of being in the summary, but were not able to produce more informative summaries than the regular RST methods.